3-D in-vivo imaging and topography using structured light

ABSTRACT

The present invention provides systems and methods for obtaining a three-dimensional (3D) representation of one or more light sources inside a sample, such as a mammal. Mammalian tissue is a turbid medium, meaning that photons are both absorbed and scattered as they propagate through tissue. In the case where scattering is large compared with absorption, such as red to near-infrared light passing through tissue, the transport of light within the sample is described by diffusion theory. Using imaging data and computer-implemented photon diffusion models, embodiments of the present invention produce a 3D representation of the light sources inside a sample, such as a 3D location, size, and brightness of such light sources.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. application Ser.No. 10/606,976, filed Jun. 25, 2003, by Daniel G. Stearns et al. andentitled, “METHOD AND APPARATUS FOR 3-D IMAGING OF INTERNAL LIGHTSOURCES,” the content of which is hereby incorporated by referenceherein, which claims priority under 35 U.S.C. § 119(e) from co-pending(1) U.S. Provisional Application No. 60/395,357, entitled “Method andApparatus for 3-D Imaging of Internal Light Sources”, by Daniel G.Stearns, et al., (2) U.S. Provisional Application No. 60/396,458,entitled “In Vivo 3D Imaging of Light Emitting Reporters”, by Bradley W.Rice, et al. and (3) U.S. Provisional Application No. 60/396,313,entitled “3D in Vivo Imaging of Light Emitting Reporters”, by Bradley W.Rice, et al. These provisional applications were all filed on Jul. 16,2002 and are incorporated by reference for all purposes.

FIELD OF THE INVENTION

The present invention relates generally to imaging with light. Inparticular, the present invention relates to systems and methods forobtaining a three-dimensional representation of a light source or lightsource distribution inside a turbid sample, which is particularly usefulin biomedical imaging and research applications.

BACKGROUND OF THE INVENTION

Bioluminescent imaging is a non-invasive technique for performing invivo diagnostic studies on animal subjects in the areas of medicalresearch, pathology and drug discovery and development. Bioluminescenceis typically produced by cells that have been transfected with aluminescent reporter such as luciferase and can be used as a marker todifferentiate a specific tissue type (e.g. a tumor), monitorphysiological function, track the distribution of a therapeutic compoundadministered to the subject, or the progression of a disease. A widerange of applications have been demonstrated including areas ofoncology, infectious disease, and transgenic animals. In vivo imaging ofcells tagged with fluorescent reporters is a related technology that hasalso been demonstrated recently in both green fluorescent protein (GFP)and near infrared (NIR) dyes such as Cy5.5.

Photons emitted by bioluminescent cells are strongly scattered in thetissue of the subject such that propagation is diffusive in nature. Asphotons diffuse through tissue many are absorbed, but a fraction reachthe surface of the subject and can be detected. In general, absorptionin mammalian tissues is high in the blue-green part of the spectrum(<600 nm) and low in the red and NIR part of the spectrum (600-900 nm).Firefly luciferase has a rather broad emission spectrum ranging from500-700=m, so at least part of the emission is in the low absorptionregion. Since the mean-free-path for scattering in tissue is short, onthe order of 0.5 mm, photons from deep sources are scattered many timesbefore reaching the surface. Bioluminescent imaging systems effectivelyrecord the spatial distribution of these photons emitted from thesurface of the subject.

However, the most important quantitative information is not directlyrelated to the surface emission but instead pertains to thebioluminescent source inside the subject. Important parameters are thesource strength (related to the number of light emitting cells),position and geometry. Most of the bioluminescent imaging work publishedto date involves use of single-view 2D imaging systems. Image analysisusually involves quantifying a light emitting region-of-interest (ROI)on the subject surface. While this analysis methodology is simple andprovides a good relative measure of light emission, it does not takeinto account the source depth and resulting attenuation through tissue.

Hence, there is interest in developing both improved imaging systems andreconstruction algorithms that would provide the three-dimensionaldistribution of photon emission inside the sample (e.g., animal) fromimages measured on the sample surface.

SUMMARY OF THE INVENTION

The present invention provides systems and methods for obtaining athree-dimensional (3D) representation of one or more light sourcesinside a sample, such as a mammal. Mammalian tissue is a turbid medium,meaning that photons are both absorbed and scattered as they propagatethrough tissue. In the case where scattering is large compared withabsorption, such as red to near-infrared light passing through tissue,the transport of light within the sample is described by diffusiontheory. Using imaging data and computer-implemented photon diffusionmodels, embodiments of the present invention produce a 3D representationof the light sources inside a sample, such as a 3D location, size, andbrightness of such light sources.

In one aspect, the present invention relates to a method for obtaining athree-dimensional representation of a light source distribution locatedinside a turbid sample. Surface light image data is provided from lightemitted from a surface of the sample originating from the light sourcedistribution located inside the sample. Light data internal to thesample surface is then obtained based on this provided surface lightimage data. In a specific embodiment, the surface light image data isconverted into photon density just below the sample surface, and thisphoton density is then used to obtain the light data internal to thesample surface to thereby result in a three-dimensional representationof the light source distribution within the sample.

In yet another aspect, the present invention relates to an imagingsystem for obtaining a three-dimensional representation of a lightsource located inside a sample. The imaging system comprises an imagingchamber having a set of walls enclosing an interior cavity. The imagingchamber also includes a camera mount configured to position a camera anda moveable stage apparatus. The movable stage apparatus includes atransport mechanism and a stage configured to support the sample withinthe interior cavity. The stage is coupled with the transport mechanismfor movement of the sample to one of a plurality of positions in theinterior cavity. The imaging chamber further includes a light transportmechanism for transmitting light emitted from a surface of the sample.The imaging chamber also includes a processor designed or configured toprovide surface light image data from light emitted from the surface ofthe sample originating from the light source distribution located insidethe sample. The processor is further configured to obtain light datainternal to the sample surface to obtain a three-dimensionalrepresentation of the light source distribution based on the surfacelight image data.

In still another aspect, the present invention relates to a computerprogram product. The computer program product comprises a computerreadable medium and program instructions provided via the computerreadable medium. The program instructions comprise reconstructioninstructions for obtaining a three-dimensional representation of a lightsource located inside a sample. The reconstruction instructions arecapable of performing one or more of the above described methodoperations.

These and other features of the present invention will be described inmore detail below in the detailed description of the invention and inconjunction with the following figures.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements and in which:

FIG. 1 is a perspective view of an imaging system adapted to produce a3D representation of a light source located inside a sample inaccordance with one embodiment of the invention.

FIG. 2A is a cut away perspective view of the chamber of FIG. 1 havinginternal components for facilitating multiple views of the sample inaccordance with one embodiment of the present invention.

FIG. 2B is a perspective view of the internal components of FIG. 2A forfacilitating multiple views of the sample in accordance with oneembodiment of the present invention.

FIG. 2C shows four different example positions of the stage relative tothe light transport device of FIGS. 2A and 2B: 0 degrees, 90 degrees,180 degrees, and 270 degrees.

FIG. 2D shows the configuration of the structured light projector usedto obtain the subject surface topography

FIG. 3A shows a simplified illustration of light imaging in a turbidmedium in accordance with one embodiment of the present invention.

FIG. 3B illustrates a schematic diagram showing a planar approximationat the surface boundary.

FIG. 3C illustrates the relationship between a volume element (voxel)and a surface element for a cylindrical object.

FIG. 4A illustrates exemplary relationships for converting between 2Dcamera data and surface data for an exemplary surface element.

FIGS. 4B-4H illustrate pictorial representations of structured lightimaging corresponding to the process flow of FIG. 5C.

FIG. 5A illustrates a process flow for obtaining a 3D representation ofone or more light sources inside a sample in accordance with oneembodiment of the present invention.

FIG. 5B illustrates a process flow for using the imaging system of FIG.1A to obtain imaging data in accordance with one embodiment of thepresent invention.

FIG. 5C illustrates a process flow for using the imaging system of FIG.1A to obtain surface topography data in accordance with one embodimentof the present invention.

FIG. 5D illustrates a process flow for converting surface light emissiondata to light internal to the surface in accordance with a specificembodiment of the present invention.

FIGS. 5E and 5F represent reconstruction results for one point-likelight source within a phantom mouse (e.g., plastic mouse having anembedded fiber optic).

FIG. 6 illustrates an exemplary computer system in accordance with oneembodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following detailed description of the present invention, numerousspecific embodiments are set forth in order to provide a thoroughunderstanding of the invention. However, as will be apparent to thoseskilled in the art, the present invention may be practiced without thesespecific details or by using alternate elements or processes. In otherinstances well known processes, components, and designs have not beendescribed in detail so as not to unnecessarily obscure aspects of thepresent invention.

1. Imaging System

The 3D light reconstruction techniques of the present invention may beimplemented within any suitable system. Several embodiments of imagingsystems in which are suitable for implementing the techniques of thepresent invention are described further in U.S. patent application Ser.No. 09/905,668 filed by Nilson et al. on Jul. 13, 2001, entitledMULTI-VIEW IMAGING APPARATUS. The entire disclosure of this applicationis incorporated herein by reference for all purposes.

FIG. 1 illustrates an imaging system 10 adapted to produce a 3Drepresentation of a light source located inside a sample in accordancewith one embodiment of the present invention. The imaging system 10 maybe used for a variety of tasks. By way of examples, these tasks mayinclude capturing photographic, luminescent, and structured lightimages. These tasks may also include performing various processing andimaging tasks such as obtaining, analyzing and manipulating 3D lightsource representations.

The system 10 generally images and processes low intensity light sourcesusing a light sensitive camera. Light from the light source refers tophotons and electromagnetic energy anywhere in the visible tonear-infrared (NIR) part of the spectrum in the wavelength range of400-950 nm. It is understood that some intensities imaged and processedin system 10 are not readily detectable by human vision. For example,low intensity light emitted from a sample may have a surface radiancebetween about 10³ to about 10¹⁰ photons/second/centimetersquared/steradian, where the lower end of this range is typically wellbelow human detection levels.

In the illustrated embodiment, the imaging system 10 includes an imagingchamber 12 adapted to receive a light-emitting sample in which lowintensity light, e.g., luciferase-based luminescence, is to be detected.A high sensitivity camera 20, such as an intensified or a charge-coupleddevice (CCD) camera, is coupled with the imaging chamber 12. The camera20 is capable of capturing luminescent, photographic (i.e., reflectionbased images) and structured light images of the sample within imagingchamber 12. The camera 20 may optionally be cooled by a suitable sourcesuch as a refrigeration device 22 that cycles a cryogenic fluid throughthe camera via conduits 24.

An image processing unit 26 optionally interfaces between camera 20 anda computer 28 through cables 30 and 32, respectively. The computer 28,which may be of any suitable type, typically comprises a main unit 36that contains hardware including a processor, memory components such asrandom-access memory (RAM) and read-only memory (ROM), and disk drivecomponents (e.g., hard drive, CD, floppy drive, etc.). The computer 28also includes a display 38 and input devices such as a keyboard 40 andmouse 42. The computer 28 is in communication with various components inthe imaging chamber 12 via cable 34. Alternatively, a computer may beintegrated with the imaging equipment in a single unit.

To provide control and communication for components in system 10,computer 28 may be implemented with any suitable combination of hardwareand software for controlling and monitoring any of the devices in system10. Components controlled by the computer 28 may include camera 20, themotors responsible for camera 20 focus, the camera lens, f-stop, one ormore motors coupled to a moveable stage included in chamber 12, etc.Computer 28 also communicates with a display 38 for presenting imaginginformation to the user and also acts an interface to control theimaging system 10.

Computer 28 also includes suitable processing hardware and software forimage processing and data manipulation as described herein. For example,computer 28 may be configured with software to build a 3D representationof a light source using light emission measurements from the surface ofa sample. In addition, the computer 28 may be configured to produce 3Dsurface topography using 2D structured light images taken from one ormore positions of the stage in the interior cavity. These 3D lightconstruction techniques are further described below. Alternatively, thecomputer 28 may be integrated with the other components of the imagingsystem 10 to form a stand alone device.

In one implementation, the 3D light reconstruction techniques discussedfurther below require bioluminescent surface radiance and surfacetopography measurements over the entire sample surface. Thus, theimaging system also includes mechanisms for capturing images of thesample surface from a variety of views and positions. Preferably theimaging system meets the following goals. First, it is desirable for thecamera to remain fixed, because the various cables and supply lines forcooling the camera are too cumbersome to move during imaging. Second,the sample preferably remains horizontal during imaging. Finally, theimaging system is preferably configured to obtain a 3D surfacetopography of the sample as input to the reconstruction techniquesdescribed further below. Any suitable combination of hardware andsoftware may be used to implement these preferred goals.

FIGS. 2A and 2B illustrate various internal components of chamber 12 forfacilitating multiple view imaging in accordance with one embodiment ofthe present invention. FIG. 2A is a cut away perspective view of chamber12 and its internal components, while FIG. 2B is a perspective view ofthe internal components of FIG. 2A in accordance with one embodiment ofthe present invention. In the illustrated embodiments, the sample isplaced on a horizontal transparent platform that rotates in conjunctionwith a mirror that turns about a horizontal optical axis so as tocapture light from the sample from various views and direct the capturedlight towards a camera for imaging.

As shown, a moveable stage apparatus is disposed in the interior cavityof chamber 12 (FIG. 2A) in the form of a stage 204 to support thelight-emitting sample 106 and a transport mechanism 202 to move thestage. A light transport device 120 is included to collect light fromthe sample at its various positions. In the illustrated embodiments ofFIGS. 2A and 2B, light emitted from the sample in the form of light beam252 a generally reflects off light transport device 120 as beam 252 b.The light transport device directs this reflected light 252 b throughaperture 122 and through lens 262 (FIG. 2B) for image capture by adetector 264 (FIG. 2B) of camera 20 (FIG. 1).

In one implementation, light transport device 120 includes an angledmirror 121 that reflects light from the sample 106 on stage 204 throughaperture 122. Outer wall 123 is substantially cylindrical and includesaperture 122 that enables light to pass from the sample 106 on stage 204via mirror 121 to imaging lens 262 (FIG. 2B). Outer wall 123 of lighttransport device 120 also prevents residual light in interior cavity ofchamber 12 not directly associated with the current viewing angle ofstage 204 from reaching lens 262. This is partially performed byconfiguring mirror 121 to be sufficiently long to span the length ofstage 204. As the stage is positioned at various positions about thestationary axis of the light transport device 120, outer wall 123 andmirror 121 cooperate to collect light primarily from the angulardirection of stage 204 which is then reflected towards lens 262.

In the illustrated embodiment, light transport device 120 is rotablycoupled to chamber 12 and rotates about a fixed axis in direction 258.The moveable stage apparatus is capable of two degrees of freedom toposition the stage 204 (and sample 106) at a plurality of positionsrelative to the light transport device 120 so as to capture light fromdifferent sides of the sample to thereby form multiple view images ofthe sample. The moveable stage apparatus allows an image of the sample,or portions thereof, to be captured by a camera from different views,angles, and positions within the imaging chamber without repositioningthe posture of the sample relative to the stage 204. FIG. 2C shows fourdifferent example positions of the stage relative to the light transportdevice (120): 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

The movable stage apparatus may be formed from any suitable number,type, and arrangement of components for achieving multiple positioningand viewing of the sample. Several embodiments of a multiple-viewimaging systems are described in the above referenced patent applicationby Nilson et al, which embodiments are incorporated herein by reference.Preferably, the movable stage apparatus also includes sensor mechanismsfor sensing a position of the sample to be used in subsequent dataanalysis procedures as described further below.

In one embodiment, a light source is provided within the barrel ofmirror assembly 120 (not shown) to illuminate the sample or specimen inthe imaging chamber 12. The light source may be continuously illuminatedor flashed to capture photographic images of the sample and is turnedoff when capturing luminescence images.

A structured light illuminator is preferably integrated into the systemfor reconstructing a surface topography of the sample. In a generalimplementation, the imaging system preferably includes a mechanism forprojecting a grid of lines onto the sample from an angle, e.g., from 20to 30 degrees from normal. The grid of lines are displaced, or phaseshifted relative to the platform, when they encounter an object withfinite height. Using conventional equations for structured lightanalysis, the surface topography data for the sample over its entiresurface can be determined from the phase shift of the lines. The linespacing may vary based on the sample surface texture and sample size,but line spacing in the range of 0.5 to 2 lines per mm may be suitablefor a mouse sample. Closer line spacing provides higher resolution, butthe lines are more difficult to track on rough surfaces such as fur. Thesurface topography data is used to build a 3D representation accordingto the techniques described below, e.g., to set boundary conditions forthe 3D reconstruction.

In a specific embodiment, a simple Kohler projection scheme is used asthe structured light source. In this case, the ruling may be illuminatedby a diffuse LED source and the ruling is then projected onto the animalstage with a magnification of approximately 10×. An example of thissystem as incorporated in system 10 is shown in FIG. 2D. The projectormodule 170 rides on the back of the rotating mirror assembly 120, sothat lines are always projected on the sample 106 at all viewing angles.The illumination pattern is projected horizontally and reflects off of asmall mirror 173 at the base of the larger turning mirror to illuminatesample 106.

Such an imaging system as described with respect to FIGS. 2A-2D isavailable from Xenogen Corporation of Alameda, Calif. Although FIGS.2A-2D illustrate one system for obtaining data from multiple positionsof sample 106 relative to camera 20, the present invention is notlimited to such a system.

2. 3D Source Reconstruction

In many turbid samples such as a mammalian subject, light traveling froma light source inside the sample and emitted from the sample surface isscattered in the sample interior (e.g., the mammal tissue) such thatlight propagation through the sample is diffusive in nature (FIG. 3A).As the light diffuses through the sample, some of the light is absorbed,but a fraction of the light reaches the sample surface.

Emission of light from a sample surface is generally specified in unitsof radiance, defined as photons/sec/cm²/steradian. The imaging systemdescribed herein is calibrated to report surface intensity in units ofradiance. The surface radiance can be related to the photon density justinside the sample surface, using a model for photon propagation at thetissue-air interface. The photon density just inside the surface canthen be related to the distribution of light emitting reporters insidethe sample using a diffusion model. Thus, the present invention relatesthe surface radiance of a turbid sample measured with an imaging systemto the distribution of light emission inside the sample. Morespecifically, the present invention produces a 3D representation of aninternal light source using reconstruction techniques that utilize thelight data emitted from the sample surface. The reconstructiontechniques employ an input data set that consists of a) a topographicalsurface representation of the sample, and b) a set of measurements (e.g.surface images) of the light radiance over at least a portion of thesurface. To facilitate processing, the surface representation may bedivided into surface elements and the interior of the sample may bedivided into volume elements or voxels that constitute a volume elementmesh. The light source distribution within the sample is described byelements of the volume element mesh.

FIG. 3A shows a simplified illustration of light imaging in accordancewith one embodiment of the present invention. FIG. 3A displays amammalian sample 302 being imaged by an imaging system 10 that uses CCDcamera 20 for light data capture. The nature of mammalian tissue andmany other turbid media causes photon scattering 307 for light travelingfrom a light source 305 within the sample to the sample surface 309,resulting in diffuse images on surface 309. Using photon diffusionmodels and imaging data, the present invention produces the 3D location,size, and brightness of light source 305 from one or more surface 309images. For user convenience, the 3D representation may be expressed asa pictorial depiction, e.g., via display 38 in FIG. 1.

One useful application of the invention is to reconstruct one or morebioluminescent or fluorescent light sources, or a source distribution,inside a living animal subject, such as a mouse, using one or moreimages of the surface light emission. This finds use in oncology,infectious disease research, gene expression research, and toxicology,for example. The present invention is suitable for use with sampleshaving a complex surface, such as an animal subject (e.g. a mouse). Asthe term is used herein, a complex surface is any surface that cannot bedescribed solely using a single polygonal description. Thereconstruction techniques described herein place no restrictions on thesource distribution, such as the number of light sources in the sampleor the sizes and shapes of the sources, and no restrictions on thegeometry, size or shape of the surface.

Light data internal to the sample surface generally refers tomathematical representation or approximation of actual light within thesample interior. This may include a set of points or volume elements,each characterized by 3D position and a source strength. In oneembodiment, the present invention divides the sample interior intovolume elements where each volume element is considered to contain apoint light source at its center. A solid mesh of these volume elementsthen defines a collection of point sources used to approximate theactual light source distribution within the sample. For example, a solidmesh of cubic volume elements is suitable.

FIG. 3C illustrates an illustrative cylindrical object that is dividedinto a solid mesh of volume elements, one of which is shown. Each volumeelement contains a point light source 314, S_(i), that contributes tothe photon density 316, ρ_(j), at the surface. For a cubic volumeelement representation, the volume element size may vary according tothe size of the sample and a desired solution accuracy. In general,increasing the solid mesh density improves the accuracy of the internallight data representation. It is possible to have the density of thesolid mesh vary with the volume of the sample. In particular, it isdesirable to have the solid mesh density increase near the light sourceto provide a more accurate representation in this region, while densityof the solid mesh decreases in areas where no activity of interest istaking place (in areas corresponding to no or minimal light generation).

The surface representation refers to the surface topography of thesample and is embodied by a mathematical description or approximation ofthe actual surface. The surface representation need not include theentire sample, and may include a portion of the sample relevant to aparticular imaging scenario. With a mouse for example, the surfacerepresentation might not necessarily include the entire mouse such asdistal portions of the tail and distal portions of every foot. Thus, thesurface representation is meant to broadly refer to any surface portionof the sample and not necessarily the entire sample. Typically, thesurface representation includes one or more surface elements or regionsof interest on the sample that produce surface light emission datarelated to the internal light source. For user convenience, the surfacerepresentation is often displayed in a pictorial depiction such as a 3Ddepiction derived from structural light construction techniques.

The sample containing the light source comprises a turbid interior ofsome sort. A turbid interior in this case refers to a volume that doesnot allow unimpeded transport of light. The turbid interior may compriseone or more mediums, structures, solids, liquids, gases, etc. In oneembodiment, the sample is modeled as homogeneous such that eachrepresentative volume element in the sample is characterized byidentical light transport properties. In another embodiment, the sampleis represented as heterogeneous such that various representative volumeelements in the sample are characterized by different light transportproperties. In a mammalian sample such as a mouse for example, theinterior may comprise a mixture of tissues, bones, organs, etc., each ofwhich may be characterized by separate light transport properties in aheterogeneous model.

The present invention presents several methods for reconstructing athree-dimensional light source distribution internal to a sample basedon the surface light emission, as will be described in greater detailbelow. Briefly, a set of structured light and luminescent images arefirst acquired, e.g., obtained with a system as described above. Thesurface topography of the animal is reconstructed using structured lightalgorithms and a surface mesh is generated. Let the surface elements beenumerated with the integers j. Using one or more luminescent images,the orientation of the imaging system, and the orientation of thesurface mesh, the photon density, ρ^(j), just below the surface of thej^(th) element can be determined. The photon density just below thesurface is related to the light intensity emitted from the surface(Eq. 1) as will be detailed below. A set of volume elements can beconstructed in the volume interior to the surface. Let the volumeelements be enumerated with the integers i. The source strength in eachvolume element, S_(i), is linearly related to the photon density justbelow the surface through a Green's function kernel, G_(ij), yielding aset of linear equations (Eq. 4) as will be detailed below.

FIG. 5A illustrates a process flow 500 for obtaining a 3D representationof one or more light sources inside a sample in accordance with oneembodiment of the present invention. Processes in accordance with thepresent invention may include up to several additional steps notdescribed or illustrated herein in order not to obscure the presentinvention.

Process flow 500 uses 3D reconstruction techniques that manipulate lightdata emitted from a sample surface. The reconstruction techniques employdata that provides information about the geometry of the sample and thespatial distribution of the light emission from the sample surface. Thismay include data that comprises a) a surface representation of thesample, and b) a set of measurements (e.g. images) of the light emissionover at least a portion of the surface. One format for this datacomprises (1) a surface representation defining the surface of thesample, (2) a set of light emission images from the sample surface, and(3) a set of parameters that define the spatial relationship between thesample and each image.

Process flow 500 begins by obtaining input data (502 and 504) used inthe 3D reconstruction techniques. For example, process flow 500 maybegin by obtaining a surface representation (502). To facilitateprocessing, the surface representation may be divided into surfaceelements or a suitable surface mesh approximating the actual surface ofthe sample (502). For example, the surface mesh may be defined by a setof connected polygons, where each polygon is a surface element. Thenumber of surface elements may vary according to the size of the sampleand a desired solution accuracy. Also the density of surface elementsmay vary from point to point on the surface mesh. In general, theaccuracy of a surface representation is improved by increasing thenumber of surface elements.

Process flow may also obtain surface light emission data from the sample(504). For imaging system 10, the surface light emission data iscontained in images captured by camera 20. The images include a set ofmeasurements of the light emission over one or more portions of thesurface. In one embodiment, multiple images of the sample are obtainedfrom multiple views of the sample relative to the camera. In this case,each image provides a different two-dimensional view of the surfacelight emission from the sample. Multiple images may be taken to provideadditional data from multiple angles.

The image data may then be mapped back onto the surface of the sample.Since a camera produces 2D data, the image data is manipulated accordingto the geometry between the sample surface and the camera lens to derivevalues of the light emission intensity (or radiance) at the surface.

FIG. 4A is an illustrative relationship for converting 2D camera data tosurface data for an exemplary surface element 402. FIG. 4A shows therelationship between surface element 402, image 404, and an entrancepupil 406 of the imaging system. The light emitted from surface element402 passes through entrance pupil 406 and is recorded in image 404. Theangle of emission with respect to the surface normal is θ₂. The entrancepupil 406 subtends a small solid angle dΩ. Imaging system 10 may collectlight emitted from a surface element 402 on the sample at an angle θ₂(measured with respect to the normal to surface element 402) into thesolid angle dΩ subtended by the entrance pupil. This information maythen be used to convert image data obtained by the camera into thesurface emission intensity corresponding to the surface geometry.

Process flow 500 then converts the surface light emission data intolight data internal to the surface to obtain the three-dimensionalrepresentation (506). The surface emission intensity is related to thephoton density just inside the sample surface. The present inventionthus converts values of light emission intensity for each surfaceelement into photon density just inside the surface. Referring again toFIG. 4A, the value of emission intensity at a surface element, I(θ₂), isrelated to the photon density p beneath the surface element. The exactform of the relationship depends on the model used to describe thetransport of photons across the surface boundary. One embodiment of thisrelationship, based on the partial-current boundary condition [Haskell,et al.], is given by: $\begin{matrix}{{I\left( \theta_{2} \right)} = {\frac{c}{4\pi\quad n^{2}}{T(\theta)}\cos\quad\theta_{2}{\mathbb{d}\Omega}\left\lfloor {1 + {\frac{3}{2}\frac{1 - R_{eff}}{1 + R_{eff}}\cos\quad\theta}} \right\rfloor\rho}} & (1)\end{matrix}$Here, c is the speed of light, n is the index of refraction of thesample medium, T is the transmission coefficient for light exiting thesample through the surface element, and θ is the internal emissionangle, which is related to the external emission angle θ₂ throughSnell's law:n sin θ=sin θ₂  (2)

The parameter R_(eff) is the average internal reflection coefficientcalculated from the following formulae: $\begin{matrix}\begin{matrix}{R_{eff} = \frac{R_{\phi} + R_{j}}{2 - R_{\phi} + R_{j}}} \\{R_{\phi} = {\int_{0}^{\frac{\pi}{2}}{2\quad\sin\quad{\theta cos}\quad\theta\quad{R(\theta)}{\mathbb{d}\theta}}}} \\{R_{j} = {\int_{0}^{\frac{\pi}{2}}{3\quad\sin\quad{{\theta cos}\quad}^{2}\theta\quad{R(\theta)}{\mathbb{d}\theta}}}} \\{{R(\theta)} = \left\{ \begin{matrix}{{\frac{1}{2}\left( \frac{{n\quad\cos\quad\theta_{2}} - {\cos\quad\theta}}{{n\quad\cos\quad\theta_{2}} + {\cos\quad\theta}} \right)^{2}} + {\frac{1}{2}\left( \frac{{n\quad\cos\quad\theta} - {\cos\quad\theta_{2}}}{{n\quad\cos\quad\theta} + {\cos\quad\theta_{2}}} \right)^{2}}} & {{{for}\quad\theta} < {\arcsin\left( {1/n} \right)}} \\1 & {{{for}\quad\theta} > {\arcsin\left( {1/n} \right)}}\end{matrix} \right.}\end{matrix} & (3)\end{matrix}$

Thus, the internal reflectivity R_(eff) depends on the index ofrefraction of the medium underneath a surface element. In tissue forexample, R_(eff) is typically in the range of 0.3-0.5.

Eqs. (1) and (2) may thus be used to convert surface emission datameasured at each surface element to values of the photon density beneaththe surface. As the term is used herein, the subscript j enumerates aset of surface elements. ρ_(j) is then the value of the photon densitycalculated at the jth surface element. Further description of theconversion between surface light emission data into light data internalto the surface to obtain the three-dimensional representation isprovided in FIG. 5D.

FIG. 5B illustrates a process flow 520 for using imaging system 10 ofFIG. 1 to obtain imaging data in accordance with one embodiment of thepresent invention (504 from process flow 500). Process flow 520 beginsby placing a sample such as a specimen or assay to be imaged for lightemission on stage 204 within imaging chamber 12 (521). Using computer28, a user inputs a desired position for stage 204. Alternatively, thedesired position for stage 204 is already known based on an automateddata collection method. Transport mechanism 202 moves stage 204 to thedesired position according to a control signal provided by computer 28(522). Light transport device 120 may also re-position according to acontrol signal provided by computer 28. Imaging chamber 12 andassociated image components are then prepared for photographic imagecapture of the sample (523). Preparation may include launching imagingand acquisition software (e.g., “LivingImage” as provided by XenogenCorporation of Alameda, Calif.) on computer 28 and initializing camera20. Further preparations may include closing door 18, activating thephotographic capture option in the software, focusing camera 20 to aspecific depth of the sample or animal, and turning on the lights inchamber 12. Preparations may also include focusing the lens of camera20, selectively positioning an appropriate the lens filter of camera 20,setting the f-stop of camera 20, etc.

A photographic image is then captured (524). In an alternativeembodiment, a “live mode” is used during photographic imaging of thesample to observe the sample in real time. The live mode includes asequence of photographic images taken frequently enough to simulate livevideo. Upon completion of photographic capture, the photographic imagedata is transferred to an image processing unit 26 and/or a processor incomputer system 28. These may be used to manipulate and store thephotographic image data as well as process the data for display oncomputer monitor 38.

In one embodiment, the present invention uses structured light duringimage capture. Thus, with stage 204 at a desired position, a structuredlight image may be captured (526). Structured light image capture may beaccomplished using a structured light projection system as describedabove, and may also include preparation of the structured lightprojection system (525) and any other components of imaging system 10used in the structured light image capture. Upon completion, thestructured light image data is transferred to an image processing unit26 and/or a processor in computer system 28, which may be useful inbuilding a 3D representation of the sample.

Subsequently, with stage 204 at a desired position or at the sameposition as a previously taken photographic or structured light image,the imaging apparatus 10 is prepared for luminescence image capture(527). Such preparation may include selecting luminescent exposure timeand binning level using the computer 28, and turning off the lightsinside chamber 12. When ready, the CCD camera 20 then captures (514) theluminescence image over a set period of time (up to several minutes).The luminescence image data are transferred to the image processing unitand/or a processor in computer 28.

At this point, a user may manipulate and store the luminescence imagedata as well as process it for display on the computer display 38. Themanipulation may also include overlaying the luminescent image with thephotographic image and displaying the two images together as a 2-D“overlay” image, with the luminescence data typically shown inpseudocolor to show intensity. As mentioned, the photon emission datamay represent the specific pixels on the camera 20 that detect photonsover the duration of the image capture period. This overlay image maythen be the basis for user analysis; and may be analyzed and manipulatedas desired. In particular, an analysis may include a summation of theillumination magnitudes over the pixels within a portion of theluminescence representation. Note that although the discussion willfocus on a single luminescence representation for the overlay image, theprocess flow 520 may include taking multiple luminescencerepresentations from the same position of stage 204, e.g., at the sametime or a later time (530).

Stage 204 may then be moved to a second position (531). While the stageis at the second position, one or more photographic, structured light,and/or luminescence images of the sample may be captured as describedabove. Image collection may further continue by capturing images of thesample from alternate positions and views of the sample. For 3Dreconstruction, photographic, structured light, and/or luminescenceimages of the sample may be captured from a number of positions. Forexample, image capture may occur at anywhere from 2 to 200 positions ofthe sample within the imaging chamber. The preferred number of images is8, spaced every 45 degrees. In one embodiment, this process is automatedand controlled via computer 28.

After a suitable number of structured light images have been capturedfrom various angles and stored, computer 28 may then process thestructured light data from each angle to generate a structured lightrepresentation (532). Each structured light image provides the surfacetopography for approximately the facing half of the sample.

In another embodiment, process flow 520 and imaging apparatus 10reconstruct the 3D surface topography of the sample using a sequence ofimages. By taking images from several viewing angles, e.g., about every45 degrees, the entire 3D surface of the sample can be reconstructed by“stitching” together the partial surface reconstructions obtained fromeach view. A sequence of images may then be taken at different viewingangles and used in reconstructing the sample's 3D surface topography.The 3D surface topography and image data may also be used inreconstructing the 3D location, brightness, and size of the light sourcewithin the sample. Once the images are received by processor 28, asuitable reconstruction algorithm is applied to the data to obtain the3D surface topography. As one of skill in the art will appreciate, thereare numerous algorithms for reconstructing a surface from structuredlight images. For example, the phase shift of each line at all points onthe image can be determined from a 2D Fourier transform. Such a processis described in detail in the article entitled “Fourier-transform methodof fringe-pattern analysis for computer-based topography andinterferometry,” by M. Takeda, H. Ina and S. Kobayshi, JOSA 72, 156-160(1982), which article is incorporated herein by reference in itsentirety. The actual surface height is then computed by “unwrapping” thephase map. Such a process is described in detail in the textbookentitled “Two-Dimensional Phase Unwrapping, Theory, Algorithms, andSoftware” by D. C. Ghiglia and M. D. Pritt, (John Whiley and Sons, NewYork, N.Y., 1998), which textbook is incorporated herein by reference inits entirety. Together, a structured light photographic representationof the sample and a luminescence representation of the sample may becombined to form a structured light superposition or 3D overlay image,with the luminescence data typically shown in pseudocolor to visuallycharacterize intensity.

FIG. 5C illustrates a process flow 510 for using imaging system 10 ofFIG. 1 to obtain surface topography data in accordance with anotherembodiment of the present invention (502 from process flow 500). FIGS.4B-4H illustrate pictorial representations of structured light imagingcorresponding to process flow 510.

Process flow 510 begins by imaging a structured light reference toproduce a pattern without the sample (511 and FIG. 4B). This may beperformed by applying structured light to a stage or surface that thesample rests upon before the sample is imaged. During image capture ofthe sample, the stage is moved to common locations as those used inimage capture without the sample.

Subsequently when the sample is in the imaging chamber, the sample isimaged with structured light (512 and FIG. 4C). Structured light uses aseries of lines of light that are projected down on a sample at an angle(at about 30 degrees, for example) to the surface normal. The lines bendas they pass over the sample, and the bend in the lines can be used todetermine the height of the surface at all locations that areilluminated by a structured light projector (such as structured lightprojector 170 described above). As shown in FIG. 2D, structured lightprojector 170 is attached to and rotates with light transport device120. In this case, structured light projector 170 consists of a Kohlerillumination system where a slide is illuminated by a light source andthen an image of the slide is projected onto the sample or background.The projection angle is large enough to get sufficient “bend” in thelines to achieve spatial resolution, but small enough that large shadowsare not present.

Process flow 510 then proceeds by imaging the sample without structuredlight (513 and FIG. 4D). This may correspond to photographic imagecapture (524) as described above. The phase shift of each line at allpoints on the background and sample may be determined from a 2D Fouriertransform.

The background data is then converted to a wrapped phase (514 and FIG.4E). Here, the background data is Fourier transformed and filteredbefore a wrapped phase is calculated. Similarly, the sample data isconverted to a wrapped phase (515 and FIG. 4F) by Fourier transformingand filtering the sample data, and the calculating a wrapped phase forthe sample data.

Surface topography for the sample is then calculated (516 and FIG. 4G).In this case, this is performed by “unwrapping” the phase map. Severalunwrapping algorithms are available to those of skill in the art forthis task. For example, the phase shift of each line at all points onthe image can be determined from using Fourier profilometry techniques.With these methods, a 2D Fast-Fourier transform (FFT) of the fringe data(FIG. 4D) is taken to determine the phase shift of the lines everywherein the image (FIG. 4F). Since the phase will shift by many multiples of2π for a typical object, the phase exhibits 2π jumps as seen in FIG. 4F.These phase jumps are “unwrapped” in order to determine the actualsurface.

The above processes (511-516) may then be repeated (517) from differentviews and positions. Imaging a sample from multiple views providesadditional information that helps techniques described herein provide amore accurate 3D surface rendering. The multiple images, or the partialsurfaces obtained from each view in the 3D imaging system, are thenregistered together to form a complete 3D surface (518 and FIG. 4H).Registering can be accomplished by using non-linear least squaresfitting techniques to minimize the distance between mesh elements on twosurfaces that are to be connected. Typically, the surfaces should have astarting orientation that is fairly close to the final registeredposition. In other words, only fine adjustments to the surface positionsmay be accommodated with this method. Another registration technique isto provide an absolute reference line or fiducial of some kind in theimage, which gives the absolute position of any partial surface withrespect to the stage, for example. If the absolute positioning of eachsurface is accurate enough, then the non-linear fitting method describedabove can be skipped.

FIG. 5D illustrates a process flow 540 for converting surface lightemission data to a reconstruction of the light source distributionwithin the volume of the object in accordance with a specific embodimentof the present invention. Processes in accordance with the presentinvention may include up to several additional steps not described orillustrated herein in order not to obscure the present invention.

Process flow 540 assumes possession of a surface representation for thesample and possession of light emission data from the sample. Processflow 540 begins by dividing the surface representation into surfaceelements (541). The number of surface elements will vary according tothe sample surface area and the desired solution accuracy. The number ofsurface elements should be large enough to capture photon densitydetails and variation across the surface. For example, between about 100and about 10,000 surface elements may be suitable for a mouse.

In addition, process flow 540 divides the sample interior volume intovolume elements (542). In one embodiment, each volume element isconsidered to contain a point light source at its center. A solid meshof volume elements then defines a collection of point sources used toapproximate the actual light source distribution. In some cases, thedensity of the solid mesh increases near the light source to provideincreased information in this space, while density of the solid meshdecreases in areas where no activity of interest is taking place (nolight generation or transport). In addition, as will be described belowwith respect to loop 556, the volume element size may vary duringsolution attainment according to various adaptive meshing techniques.For example, the initial volume element size may range from about 0.1cm³ to about 1 cm³, and the final volume element size for volumeelements close to the source may reduce from about 1*10⁻³ to about1*10⁻² cm³. In a specific example, the initial volume element size maybe about 1 cm³, and the final volume element size for volume elementsclose to the source may reduce to about 8*10⁻³ cm³.

Process flow 540 then establishes a relationship between the surfaceelements and volume elements (544). In one embodiment, thereconstruction method uses a linear relationship between the sourceemission strength and the photon density at the surface. In a specificembodiment, the linear relationship is described by a Green's function.The Green's function contains all of the information about the transportof photons inside the sample, including the effects of inhomogeneitiesin the volume and internal reflection at the boundary. The Green'sfunction describes the transport of photons inside the sample from eachpoint in the source distribution to the inside of each surface elementof the sample.

When the medium inside the sample is assumed or modeled as homogeneous,one useful form for the Green's function is a simplified approximationin which the surface of the sample is treated locally as a planarinterface oriented perpendicular to a line connecting a volume elementcenter and a surface element. The photon density at the surface is theanalytical solution for a point source in a semi-infinite slab using thepartial-current boundary condition. Since the solution is only afunction of the distance between the volume element and the surface, thesimplified Green's function can be calculated for all pairs of volumeelements and surface vertices with minimal computational expense.

With a linear relationship between the source strength in each volumeelement and the photon density at each surface element described by aGreen's function G_(ij), the photon density at the jth surface elementmay be approximated by the sum of the contributions from all the volumeelements: $\begin{matrix}{\rho_{j} \cong {\sum\limits_{i}{G_{ij}S_{i}}}} & (4)\end{matrix}$where the index i enumerates the volume elements and S_(i) is the valueof the strength of the point source (photons/sec) inside the ith volumeelement.

Light transport in turbid media such as tissue is dominated byscattering and is essentially diffusive in nature. The condition fordiffusive transport is that the scattering coefficient μ_(s) be greaterthan the absorption coefficient μ_(a) so that the change in the photondensity is small between scattering events. The photon density producedby a source power density, U_(i), in a homogeneous medium may berepresented by the diffusion equation:D∇ ²ρ−μ_(α) cρ=−U _(i)( x )  (5)where the diffusion coefficient D is, $\begin{matrix}{D = \frac{c}{3\left( {\mu_{A} + \mu_{S}^{\prime}} \right)}} & (6)\end{matrix}$

In this case, the Green's function is the solution to Eq. (5) subject tothe boundary condition imposed by the surface of the sample. For asample modeled as homogeneous, a useful approximate solution for theGreen's function uses a planar approximation at the surface boundary.

FIG. 3B illustrates a schematic diagram showing this planarapproximation. A plane boundary 312 is drawn through the center of thejth surface element, perpendicular to the line connecting the pointsource at x_(i) and the surface element at x_(j). The photon density inthe planar approximation is the solution for the point source at x_(i)in a semi-infinite slab defined by the plane boundary, subject to thepartial current boundary condition. In this case, the surface of thesample is treated locally as a simple planar boundary orientedperpendicular to the line connecting the source at x_(i) and the surfaceelement at x_(j). Specifically, the boundary condition is simplified tothe case of a plane boundary, although the orientation of the boundarymay change for each surface element.

This simplified Green's function is the analytical solution for a pointsource in the semi-infinite slab using the partial-current boundarycondition: $\begin{matrix}{G_{ij} = {\frac{1}{2\pi\quad D}\left\{ {\frac{\exp\left( {{- \mu_{eff}}r_{ij}} \right)}{r_{ij}} - {\frac{1}{z_{b}}{\exp\left( {r_{ij}/z_{b}} \right)}{E_{1}\left\lbrack {\left( {\mu_{eff} + \frac{1}{z_{b}}} \right)r_{ij}} \right\rbrack}}} \right\}}} & (7)\end{matrix}$Here r_(ij)=|x_(j)−x_(i)|, E₁ is the first order exponential integralandμ_(eff)=[3μ_(A)(μ_(A)+μ_(S))]  (8) $\begin{matrix}{z_{b} = {\frac{2D}{c}\frac{1 + R_{eff}}{1 - R_{eff}}}} & (9)\end{matrix}$

In the simplified model just described, the simplified Green's functiondepends only on the distance between the volume element and the surface.It is not necessary to use an analytical form such as the simplifiedapproximation to define the Green's function.

The present invention does not rely on an analytical form such as theapproximation described above. In another embodiment, a look-up tablemay define the Green's function. The look-up table may be created byprevious measurements of photon transport in a sample (or similar sampleapproximated to be substantially equal to the current sample), or bycomputational simulations using techniques such as Monte Carlo or finiteelement modeling. This particular method is useful for samplesconsisting of inhomogeneous media, such as animal subjects. In thiscase, the optical properties, μ_(a) and μ_(s)′ from Eq. 8, now havespatial dependence.

The planar boundary approximations discussed above work best for smoothsurfaces with a large radius of curvature, and for cases where theabsorption coefficient is not too small (μ_(a)>0.1 cm⁻¹). An advantageof the planar approximation technique described above is that it iscomputationally convenient for solving the diffusion equation with anarbitrary complex boundary such as a mouse. Areas with more structure,such as the head or the limbs of a mouse, may benefit from a moreaccurate model of the boundary. Using a finite element modeling code tocalculate the Green's functions is one option to obtain a more accurateboundary model. Finite element codes such as Flex PDE, from PDESolutions, Inc. may be used for example. Another option will be toextend the planar surface approximation to first order in curvature,which may allow continued use of analytic expressions for G_(ij).

Once the Green's function is determined, the reconstruction is obtainedby solving the system of linear equations that relate the photon densityat the surface to the source distribution inside the object. Processflow 540 then proceeds by solving for all the internal volume elements(546). More specifically, given the modeling described above, thereconstruction techniques solve the system of linear equations thatrelate the photon density at the surface to the source distributioninside the sample. Thus, once the Green's function is determined, it maybe evaluated for every volume element—surface element pair, in order toobtain the system of linear equations (Eq. 4). The final step of thereconstruction method is to solve the linear system, Eq. (4), for thesource strengths S_(i). Referring back to Eq. (4), since ρ is known, andG_(ij) can be determined as described above, the reconstructiontechniques then solve for S_(i). Typically, there is no exact solutionto the linear system because the collection of point sources is only anapproximation of the actual source distribution. One suitablereconstruction is then the best approximate solution of the linearsystem.

In one embodiment, the linear system is converted into an optimizationproblem to find an approximate solution. In this case, a cost functionis used to define the optimization problem. One suitable cost functionmay be represented by the sum of the source strengths: $\begin{matrix}{{Cost} = {\sum\limits_{j}S_{j}}} & (10)\end{matrix}$

The cost function is subject to one or more linear constraints. A firstsuitable set of constraints is that the source strengths be positivedefinite:S_(i)≧0  (11)

A second suitable set of constraints is a system of linear inequalitiesbased on the system of linear equations (4). It requires that thecontributions to the photon density summed over all the volume elementsbe less than the measured value of the photon density at each surfaceelement: $\begin{matrix}{{\sum\limits_{j}{G_{ij}S_{j}}} \leq \rho_{i}} & (12)\end{matrix}$

In a specific embodiment, an optimum solution for source strengths S_(i)is found by maximizing the cost function (10) subject to constraints(11) and (12).

Process flow 540 then solves for the current optimal solution in the setof solutions (550). The optimization problem described above may besolved using linear analysis techniques. One particularly efficienttechnique is the SIMPLEX method.

The solution quality may be assessed (552). In one embodiment, theassessment measures the difference between the observed photon densityand the calculated photon density. For example, a “chi squared” criteriamay be used: $\begin{matrix}{\chi^{2} = {\sum\limits_{j}\left\lbrack \frac{\rho_{j} - {\sum\limits_{i}{G_{ij}S_{i}}}}{\rho_{j}} \right\rbrack^{2}}} & (13)\end{matrix}$The value of χ² measures the difference between the observed photondensity ρ_(i) and the calculated photon density$\sum\limits_{i}{G_{ij}S_{i}}$over the surface of the sample.

Determination of internal 3D light source distribution data based onsurface light emission data may be embedded in one or more iterationloops. Iteration may allow for refinement of the volume mesh and mayallow searches for the optimal sampling of the surface elements and theoptimal weighting function configuration. In general, the convergence ofthe reconstruction is improved by reducing the size of the linearsystem. In one embodiment, reconstruction techniques of the presentinvention iteratively vary one or more of: the volume elementconfiguration, the surface element configuration, and/or the costfunction configuration.

Iteration may also assist solution attainment. The SIMPLEX methodconverges rapidly to a solution, and hence has the advantage of being acomputationally efficient algorithm. To assist the SIMPLEX algorithm infinding the global maximum, and hence the optimum solution, the SIMPLEXcalculation may be repeated numerous times while systematically varyingkey elements in the algorithm (554). This can be achieved by usingadaptive meshing, or adaptive gridding, to vary the number of volumeelements, and by using a subset of the surface elements.

Loop 556 (FIG. 5D) illustrates repeating the solution steps whilevarying the number of volume elements. Typically, it is advantageous toreduce the number of volume elements in the problem while maintaining ahigh density of volume elements in the vicinity of the source. This canbe achieved by using adaptive meshing. In one suitable adaptive meshingapplication, a coarse volume element mesh is initially appliedthroughout the entire sample volume and the current solution is found(544, 546, and 550), yielding an initial solution for S_(i). Next thevolume elements that have source strengths greater than zero (S_(i)>0)are refined (i.e. subdivided) and those where the source strengths equalzero (S_(i)=0) are removed. Solution attainment and volume element meshrefinement may then be iterated repeatedly, producing a high-densityvolume element mesh localized in the source region. During eachiteration, the quality of the current solution is assessed (552). In aspecific embodiment, the iteration continues until further refinementproduces no significant decrease in the value of χ2.

An additional iterative improvement may be obtained by varying thenumber of surface elements, N_(S), used in obtaining thethree-dimensional representation (loop 555). Using a subset of thesurface elements of the surface mesh reduces the number of constraintsin the problem, which may simplify solution calculation. The number ofsurface elements may be used to sample the surface uniformly. In thiscase, process flow 540 iterates for different values of N_(S)corresponding to sampling the surface element mesh at differentdensities, and use the quality assessment (552) to determine the bestsolution among the different values of N_(S). For example, if the numberof surface elements is between about 100 and about 300 surface elementsfor a small mouse, an iteration step size between about 10 and 50 mayalso suitable.

Another iteration loop varies a cost function to improve solutionobtainment. For example, the cost function may be modified to include aweighting factor W_(i): $\begin{matrix}{{{Cost} = {\sum\limits_{i}{S_{i}/W_{i}^{\gamma}}}},{W_{i} = {\sum\limits_{j}G_{ij}}}} & (14)\end{matrix}$The weighting factor W_(i) is the contribution of the i^(th) volumeelement to the photon density over the entire surface. The exponent γadjusts the relative contribution to the cost function of the interiorvolume elements and those volume elements close to the surface. Whenγ=0, then the interior volume elements have relatively greater weight.When γ=1 the volume elements near the surface have greater weight.Process flow 540 may be iterated while varying γ to search for solutionswhere the source is both near and far from the surface. For example, thestep size may be varied by about 0.01 to about 0.2 for a range of γ from0 to 1. In a specific embodiment, the step size was varied by about 0.05for a range of γ from 0 to 1. Once again, quality assessment (552) maybe used to identify the best solution.

In a specific embodiment, the SIMPLEX calculation is imbedded in asearch routine that consists of two optimization loops. The inner loopuses adaptive gridding to optimize the configuration of the volume mesh.The outer loop searches for the optimum values of the parameters N_(S)and γ. Each step of the search algorithm is tested using the chi-squaredcriteria. The global solution is defined as the one that minimizes χ².

FIGS. 5E and 5F represent reconstruction results for one point-likelight source within a phantom mouse (e.g., rubber mouse having anembedded fiber optic). FIG. 5E illustrates the reconstructed sourcedistribution within phantom mouse 702 a and 702 b at two differentangles. The surface shown represents the contour of the isosurface athalf the maximum source strength. FIG. 5F shows images of the photondensity (e.g., 708 a and 708 b) directly beneath the surface of thephantom mouse 702. The first row 706 a is the logarithm of the measuredphoton density. The second row 706 b is the logarithm of the simulatedphoton density, calculated from the reconstructed source distribution.The third row 706 c is the difference of the first and second rows.

Although process flows 500, 510, 520 and 540 have been described withmany simplifications to expedite processing, the present invention isnot limited by these simplified computational methods. For example, theGreen's Function may be calculated without many of the simplificationsdescribed above, even at the cost of increased computationalrequirements. In addition, while process flows 500, 510, 520 and 540describe a specific method of obtaining measurements of light emissionfrom the sample based on the system illustrated in FIGS. 1-2E, 3Dreconstruction techniques of the present invention are not limited tohow the 2D light emission data is obtained or to the use of anyparticular apparatus. Indeed, other imaging apparatus other than thatdescribed above with respect to FIGS. 1-2E may be used, such as animaging apparatus described in commonly owned pending patent applicationSer. No. 09/905,668, which was previously incorporated by reference. Inaddition, 2D light emission data may be obtained from an independentsource and stored as data within computer 28, and not necessarilyproduced as the result of imaging via a complementary or local imagingsystem.

Further, although the present invention has been described so far withrespect to a bioluminescent source, the present invention may be used toobtain 3D reconstructions of any type of light sources, including one ormore fluorescent sources. For a fluorescent source, the relationshipbetween the surface elements and the volume elements may be altered(544). For example, a Green's function in the linear system may bemultiplied by a second Green's function that describes the transport ofthe excitation light from the sample surface to the volume elements. Ina specific approximation, the Green's function in the linear system (4)may be modified to be the product of two Green's functions:G _(ij) =G _(i) ^(E) G _(ij) ^(F)  (15)The first Green's function, G_(i) ^(E), describes the transport ofexcitation light from the excitation source at the surface of the sampleto the i^(th) volume element. The second Green's function, G_(ij) ^(F),describes the transport of the fluorescent light from the i^(th) volumeelement to the j^(th) surface element. Both Green's functions can bedetermined from analytical expressions, such as the simplifiedapproximation described above in the case of a homogeneous medium, orfrom look-up tables in the case of an inhomogeneous medium. Theexcitation and fluorescent light are typically at different wavelengths,and thus the fluorescence does not stimulate additional fluorescence.The system of linear equations (4) is still valid, and process flow 540can be used as outlined above to determine the fluorescent light sourcedistribution.

The present invention is also well suited to handle noise in the inputimages. Any practical imaging system will have a limited dynamic rangeand be subject to noise sources. This may compromise the fidelity of theimages used as input data in the reconstruction, and consequently maydegrade the quality of the 3D representation. To this end, the presentinvention may also include techniques designed to improve itsperformance when the input images are noisy and/or have a limiteddynamic range.

A limited dynamic range is particularly evident when imagingbioluminescence from sources imbedded in tissue, because the lightemission intensity typically varies over many orders of magnitude acrossthe sample surface. If the imaging camera imposes a limited dynamicrange and a region of highest intensity is set to the camera's upperlimit, then there will probably be regions in the image where theemission intensity falls below the bottom of the dynamic range. Theseregions of the image will be received as noise; and correspond to a‘noise floor’.

The reconstruction techniques provided herein map images of surfacelight emission onto the three-dimensional sample to determine the valueof the photon density underneath the surface. With noise in the images,the reconstruction techniques map only those regions of the images thatare above the noise floor. This can be done by defining a threshold foreach image that is a fraction of the peak intensity. In one embodiment,the threshold is related to the peak intensity and dynamic range of thecamera. In a specific embodiment, the threshold may be larger than thenumber represented by dividing the peak intensity by the dynamic range.For example, if the dynamic range of the camera is 1000 and the peakintensity in the image is I_(P), than a suitable value for the thresholdmay be I_(P)/500. Only the regions of the image that are above thethreshold are mapped onto the sample; and those regions below thethreshold are not used. As a result, there will be regions of the samplesurface that are empty after mapping is completed, that is, that have nophoton density values associated with the surface mesh. Typically, thesewill tend to be the parts of the surface that are the farthest away fromthe bioluminescent source.

Process flow 540 solves the following optimization problem: maximize thetotal source strength subject to the constraint that the calculatedphoton density is less than the measured photon density everywhere onthe surface. However, process flow 540 may also account for empty partsof the surface. This is because when the photon density is unknown, theconstraints on the source strength are removed for those volume elementsnear the empty regions. Thus, there may be insufficient information todetermine the source strength in those volume elements. The modificationof process flow 540 consists of eliminating volume elements in thevolume grid that are insufficiently constrained by the existing photondensity information. This is achieved by comparing the contribution of avolume element to the photon density in the empty and non-empty surfaceregions. For example, if P and Q represent the sets of surface elementsthat consist of the empty and non-empty regions, respectively, then thecriteria for removing the i^(th) volume element from the volume grid is:$\begin{matrix}{{\sum\limits_{j \in Q}G_{ij}} < {\kappa{\sum\limits_{j \in P}G_{ij}}}} & (16)\end{matrix}$The constant κ may have a value in the range of 1-10. The criteria (16)is applied to each volume element during the formation of the initialvolume grid (542) and at each iteration, if used.

In addition to having a limited dynamic range, realistic images maycontain noise. One minimum noise contribution may be represented by theshot (Poisson) noise associated with the sampling of discrete numbers ofphotons in each image pixel. When shot noise dominates, thesignal-to-noise ratio in the image varies with the square root ofintensity. This suggests that the regions of highest intensity, orequivalently those regions of the surface having the highest photondensity, should have the relatively smallest amount of noise. To accountfor this noise, the assessment for solution quality may altered (552).For example, expression (13) may be modified to account for noise. Morespecifically, each term in the sum may be weighted by a factor of thephoton density ρ_(i) to become: $\begin{matrix}{\chi^{2} = {\sum\limits_{j}{\frac{1}{\rho_{j}}\left\lbrack {\rho_{j} - {\sum\limits_{i}{G_{ij}S_{i}}}} \right\rbrack}^{2}}} & (17)\end{matrix}$

The reconstruction techniques of the present invention will typically beimplemented by a suitable processor or computer-based apparatus.Referring to FIG. 6, an exemplary computer system 350 includes a centralprocessing unit (CPU) 352, read only memory (ROM) 354, random accessmemory (RAM) 356, expansion RAM 358, input/output (I/O) circuitry 360,display assembly 362, input device 364, and expansion bus 366. Computersystem 350 may also optionally include a mass storage unit 368 such as adisk drive unit or nonvolatile memory such as flash memory and areal-time clock 360.

Regardless of computer system 350 configuration, it may employ one ormore memories or memory modules configured to store program instructionsfor obtaining a three-dimensional representation of a light sourcelocated inside a sample and other functions of the present inventiondescribed herein. Such memory or memories may also be configured tostore data structures, imaging data, or other specific non-programinformation described herein.

Because such information and program instructions may be employed toimplement the systems/methods described herein, the present inventionrelates to machine-readable media that include program instructions,state information, etc. for performing various operations describedherein. Examples of machine-readable media include, but are not limitedto, magnetic media such as hard disks, floppy disks, and magnetic tape;optical media such as CD-ROM disks; magneto-optical media such asfloptical disks; and hardware devices that are specially configured tostore and perform program instructions, such as read-only memory devices(ROM) and random access memory (RAM). The invention may also be embodiedin a carrier wave traveling over an appropriate medium such as airwaves,optical lines, electric lines, etc. Examples of program instructionsinclude both machine code, such as produced by a compiler, and filescontaining higher level code that may be executed by the computer usingan interpreter.

It should be borne in mind that although computer system 350 isdiscussed in some detail herein to facilitate discussion, the inventionmay be practiced using a variety of suitable computer-implementedtechniques. In general, any suitable computer system may be employed forobtaining a three-dimensional representation of a light source locatedinside a sample. Further, the inventive reconstruction techniquesdisclosed herein may be implemented via a computer network, such as alocal area network (LAN), wide area network (WAN) or a global computernetwork such as the Internet. In the latter cases, the reconstructiontechniques may be implemented at least in part as downloadable computersoftware and data (e.g., applets such as JAVA™ applets from SunMicrosystems Inc.). The downloadable computer software and data may bekept on one or more servers on the network, accessible by any clientcomputer or terminal capable and authorized for such access. Networkcomputing techniques and implementations are well known in the art andare not discussed in great detail here for brevity's sake.

While this invention has been described in terms of several preferredembodiments, there are alterations, permutations, and equivalents whichfall within the scope of this invention which have been omitted forbrevity's sake. For example, images may be acquired and analyzed at oneor more different wavelengths. It is therefore intended that the scopeof the invention should be determined with reference to the appendedclaims.

1. A method for imaging an animal resting on a stage that supports theanimal, the method comprising: directing structured light onto theanimal while the animal rests on the stage; capturing a structured lightimage of the animal using a camera configured to capture an image of theanimal while the animal rests on the stage; and capturing a second imageof the animal using the camera, wherein the second image comprises datarelated to luminescent or fluorescent light emitted from within theanimal.
 2. The method of claim 1 wherein the structured light comprisesa grid of lines.
 3. The method of claim 2 wherein directing thestructured light comprises projecting the grid of lines onto the animalusing a projector.
 4. The method of claim 1 further comprising buildinga surface representation of the animal using structured light dataincluded in the structured light image.
 5. The method of claim 4 furthercomprising overlaying luminescent or fluorescent data related to theluminescent or fluorescent light onto the surface representation.
 6. Themethod of claim 4 wherein the surface representation provides a surfacetopography for a portion of the animal facing the camera.
 7. The methodof claim 1 wherein the stage is moveable and further comprisingpositioning the moveable stage at a desired position in the imagingchamber.
 8. The method of claim 1 wherein the animal is a mouse.
 9. Themethod of claim 1 further comprising imaging a structured lightreference to produce a structured light pattern without the animal. 10.The method of claim 1 further comprising capturing a photographic imageof the animal using the camera.
 11. A method for obtaining a surfacerepresentation of an animal, the method comprising: moving a moveablestage included in an imaging chamber to a first position in the imagingchamber; capturing a first structured light image of the animal from thefirst position using a camera; moving the moveable stage to a secondposition in the imaging chamber, wherein the second position has adifferent angle relative to a fixed datum associated with the camerathan the first position; capturing a second structured light image ofthe animal from the second position using the camera; and building asurface representation of the animal using structured light dataincluded in the first structured light image and structured light dataincluded in the second structured light image.
 12. The method of claim111 further comprising directing structured light onto the animal. 13.The method of claim 12 wherein the structured light directed onto theanimal comprises a grid of lines.
 14. The method of claim 13 whereindirecting the structured light comprises projecting the grid of linesonto the sample.
 15. The method of claim 14 further comprisingdetermining surface topography data for the animal from the phase shiftof the lines.
 16. The method of claim 11 wherein the animal is a mouse.17. The method of claim 11 further comprising imaging a structured lightreference from the first or the second position to produce a structuredlight pattern without the animal.
 18. The method of claim 11 furthercomprising capturing a photographic image of the animal from the firstor the second position using the camera.
 19. The method of claim 11wherein the first structured light image provides a surface topographyfor a portion of the animal facing the camera at the first position. 20.The method of claim 19 wherein the second structured light imageprovides a surface topology for a portion of the animal facing thecamera at the second position.
 21. The method of claim 20 furthercomprising combining data included in the first structured light imageand data included in the second structured light image.
 22. The methodof claim 11 further comprising capturing a second image of the animalfrom the first position using the camera.
 23. The method of claim 22wherein the second image comprises a luminescent or fluorescent data.24. The method of claim 23 further comprising overlaying the luminescentor fluorescent data onto the surface representation.
 25. A method forproducing a map of photon density on a surface representation of ananimal, the method comprising: directing structured light onto theanimal while the animal rests on the stage; capturing a structured lightimage of the animal using a camera configured to capture an image of theanimal while the animal rests on the stage; capturing a second image ofthe animal using the camera, wherein the second image comprises datarelated to luminescent or fluorescent light emitted from within theanimal; building a surface representation of the animal using structuredlight data included in the first structured light image; and mapping theluminescent or fluorescent data onto the surface representation of theanimal.
 26. The method of claim 25 wherein the structured lightcomprises a grid of lines.
 27. The method of claim 26 wherein directingthe structured light comprises projecting the grid of lines onto thesample.
 28. The method of claim 27 wherein projecting the grid of linesonto the sample comprises projecting the grid of lines onto the samplefrom an angle.
 29. The method of claim 28 further comprising determiningsurface topography data for the animal from the phase shift of thelines.
 30. The method of claim 25 wherein the stage is moveable andfurther comprising positioning the moveable stage at a desired positionin the imaging chamber.
 31. The method of claim 25 wherein the animal isa mouse.
 32. The method of claim 25 further comprising imaging astructured light reference to produce a structured light pattern withoutthe animal.
 33. The method of claim 25 further comprising capturing aphotographic image of the animal using the camera.